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Jensen's Inequality 


6.2.5 Jensen's Inequality 

Remember that variance of every random variable X is a positive value, i.e., 

Var{X) = EX 2 - (EX) 2 > 0 . 


Thus, 


EX 2 > (EX) 2 . 

If we define g(x) = X 2 , we can write the above inequality as 

E(g(X)) > g(E(X}). 

The function g(x ) - X 2 is an example of convex function. Jensen's inequality states that, for any convex function g, we have 
E(g(X)] > g(E [X] ). So what is a convex function? Figure 6.2 depicts a convex function. A function is convex if, when you pick any two 
points on the graph of the function and draw a line segment between the two points, the entire segment lies above the graph. On the other hand, 
if the line segment always lies below the graph, the function is said to be concave. In other words, g(x) is convex if and only if —g(x) is 
concave. 




Fig.6.2 - Pictorial representation of a convex function and a concave function. 


We can state the definition for convex and concave functions in the following way: 


Definition 6.3 

Consider a function g : I — y M, where I is an interval in M.. We say that g is a convex function if, for any two points X and y in I and 
any a £ [0,1], we have 

g(ax + (1 - a)y) < ag(x) + (1 - a)g(y). 

We say that g is concave if 

g(ax + (1 - a)y) > ag(x) + (1 - a)g(y). 
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Note that in the above definition the term CHX + (1 — Ot)y is the weighted average of X and y. Also, Ug(x) + (1 — Oi)g(y ) is the 
weighted average of g{x) and g(y)■ More generally, for a convex function g : I —Z IR, and X\,X 2 ,...,X n in I and nonnegative real 
numbers CL{ such that G.\ + CL 2 + . . . +Q! n = 1, we have 

g( a i x i + ot 2 x 2 +... +a n x n ) < a x g{xi) + a 2 g{x 2 )+... +a n g{x n ) ( 6 . 4 ) 

If Tl = 2, the above statement is the definition of convex functions. You can extend it to higher values of Tl by induction. 

Now, consider a discrete random variable X with n possible values X \, X 2 ,-,X n . In Equation 6.4, we can choose 

Oi{ — P(X = X {) = Px{ x i)- Then, the left-hand side of 6.4 becomes g[EX) and the right-hand side becomes E[g(X)\ (by 

LOTUS). So we can prove the Jensen's inequality in this case. Using limiting arguments, this result can be extended to other types of random 

variables. 



To use Jensen's inequality, we need to determine if a function g is convex. A useful method is the second derivative. 



For example, if g(x) = X 2 , then g"{x) = 2 > 0,thus^(x) = X 2 is convex over M. 


Example 6.24 

Let X be a positive random variable. Compare E[X a ] with [E[X]) a for all values of CL G M. 

• Solution 

o First note 

E[X a } = 1 = (E[X]) a , 

E[X a } = EX = (^[X]) a , 

So let's assume a ^ 0,1. Letting g(x) = X a , we have 

g"{x) = a(a — l)a; a_2 . 

On (0, oo), we can say g n (x) is positive, if a < 0 or CL > 1. It is negative, if 0 < CL < 1. Therefore we conclude that 
g( x ) is convex, if a < Oora > 1 . It is concave, if 0 < a < 1. Using Jensen's inequality we conclude 

E[X a ] > (E[X]) a , if a < 0 or a > 1 . 

E[X a ] < (E[X}) a , if 0 < a < 1. 


<— previous 
next —* 


if a = 0, 
if a = 1. 
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